test(scripts): add platform support-declaration consistency tests - #24
Merged
phodal merged 1 commit intoJul 30, 2026
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a contract-style Node test to keep the declared supported host/platform set consistent across key public surfaces (CLI help + gates, exported session-analysis help, agent-customize provider registry, asset-baseline provider gate, and the adapters documentation matrix), and records the work in the roadmap and a supporting spec.
Changes:
- Add
test/support-declarations.test.mjsto assert a single canonical supported-platform set matches what multiple surfaces declare. - Mark roadmap item A-06 as done.
- Add a spec documenting intent, scope, and acceptance criteria for the consistency test.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test/support-declarations.test.mjs | New contract test asserting platform declarations across CLI/help/registries/docs stay in sync |
| roadmap.md | Checks off A-06 to reflect completion of the roadmap item |
| docs/specs/2026-07-28-a06-support-declaration-consistency.md | Adds spec describing the consistency-test contract and acceptance criteria |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
creayma-del
force-pushed
the
test/support-declaration-consistency
branch
2 times, most recently
from
July 30, 2026 02:49
e3377de to
35bd93f
Compare
The supported platform set (qoder, codex, claude, cursor, qwen, copilot) is declared independently in CLI help, the agent-customize provider registry, the session-analysis platform loader, the Harness report platform gate, the asset-baseline provider gate, and the host adapter matrix. Nothing kept those declarations in agreement, so adding or dropping a host in one owner silently left the others stale. Add test/support-declarations.test.mjs, which derives the declared list from each surface (help output, registry keys, fail-closed error messages, and documented module paths) through public routes only and asserts set equality against one canonical list. To let the contract test assert the report platform gate's whitelist rather than only that it rejects unknown input, extract the reportPlatform whitelist to a file-local REPORT_PLATFORMS constant and have its rejection error name the full supported set, matching the session-analysis and asset-baseline gates. This is the only runtime change; the error prefix and error.code are unchanged. This closes roadmap P0 item A-06. Story: roadmap.md TODO A-06 Spec: docs/specs/2026-07-28-a06-support-declaration-consistency.md Test: node --test test/support-declarations.test.mjs (6 pass) Co-authored-by: QoderAI (Qwen 3.8 Max) <qoder_ai@qoder.com>
creayma-del
force-pushed
the
test/support-declaration-consistency
branch
from
July 30, 2026 03:26
35bd93f to
0750bc8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a contract test that keeps the platform support declarations named by the
roadmap Definition of Done in agreement: session-analysis CLI help and platform
gate, the exported
SESSION_ANALYSIS_HELP, the agent-customize providerregistry,
createAnalyzerrejection,harness analyzehelp and platform gate,the asset-baseline provider gate, and the host adapter matrix. Any host added
or dropped in one owner without the others now fails
npm test.To let the test assert the report platform gate's whitelist (not just that it
rejects unknown input), the
reportPlatformgate's rejection error now namesthe full supported set, matching the session-analysis and asset-baseline gates.
Why
roadmap.mdTODO A-06 (P0) — "Add support-declarationconsistency tests."
qoder,codex,claude,cursor,qwen,copilot) is declared independently in at leastfive places. Nothing asserted they agree, so adding a host in one owner
silently left CLI help, error messages, and docs advertising different
platform sets (this actually happened when
qwenand latercopilotlanded). This test closes that gap.
Traceability and Scope
docs/specs/2026-07-28-a06-support-declaration-consistency.mdcanonical set), AC-2 (public routes only: root facade CLI, the
session-analysis/index.mjsandagent-customize/providers/index.mjsimportsurfaces, shipped docs), AC-3 (unsupported input fails closed naming the full
set on each gated route), AC-4 (
npm testdiscovers the test automatically).scripts/harness-analysis/report-run.mjs—reportPlatformwhitelist extracted to a file-localREPORT_PLATFORMSconstant; its rejection error now names the supported set. Each capability
still owns its own declaration. Also adds
test/support-declarations.test.mjs,the spec, a
CHANGELOG.mdentry, and checks the A-06 roadmap box.runtime constant across modules;
ANALYZE_HELPkeeps its own prose list(asserted separately by the test); no prose-wording assertions beyond the
declared platform lists; other commands with their own platform/provider
lists (coding-agent-practices
inventory/asset-integrity,agent-customize,agent-lint,evidence-bundle,task-loop-source,selection-profile) stay out of scope.Change Type
Test and Review Evidence
node --test test/support-declarations.test.mjsnode --test test/doc-link-graph.test.mjsnode --test test/harness-report-run.test.mjs test/harness-report-quality.test.mjs test/harness-report-render-cli.test.mjs test/harness-report-source.test.mjs test/report-source-review.test.mjsgit stash(changes removed)mainand unrelated to this changeCHANGELOG.mdmain(copilothost)reportPlatformresolved by keeping the constant/error-list approach and addingcopilot; canonical set updated to 6 platforms; all 6 consistency tests passManual or visual evidence: n/a (no visual surface).
Risk and Recovery
Supported platforms: ...to thereportPlatformrejection message; theunsupported Harness report platform: <name>prefix and theUNSUPPORTED_REPORT_PLATFORMerror code are unchanged, and no other code ortest depends on the old message format. Test uses
process.execPath+spawnSync(no shell), so it runs on Windows, macOS, and Linux.new test file, the spec, or the extracted constant.
help/error phrasing (
--platform <...>,Supported platforms: ... .);rewording those messages requires updating the extraction patterns. Platform
declarations in the out-of-scope commands listed under non-goals can still
drift and are intentionally not covered.
AI Involvement
wording, approved each step; a code-review pass caught and fixed a spec
statement that overstated help/constant sharing; the branch was rebased onto
the latest
mainand the reportPlatform conflict resolved manually; thelisted test commands were executed locally with results reported verbatim
above.
Checklist
AGENTS.md,CONTRIBUTING.md, and the relevant canonical-owner guidance.CHANGELOG.md.